THE LEMPERT THEOREM AND THE TETRABLOCK 



ARMEN EDIGARIAN, LUKASZ KOSINSKI, AND WLODZIMIERZ ZWONEK 

Abstract. In the paper we show that the Lempert property (i.e. the equality be- 
tween the Lempert function and the Caratheodory distance) holds in the tetrablock, a 
bounded hyperconvex domain which is not biholomorphic to a convex domain. The ques- 
tion whether such an equality holds was posed in [3]. 



1. Introduction and main results 

The paper may be seen as a direct continuation of the study of the geometry of the 
tetrablock, a domain introduced recently and then studied in a series of papers ([3], 0, 

m, m, iei)- 

The tetrablock naturally appears in control engineering and produces problems of a 
function-theoretic character. We denote the tetrablock by E and we define it to be the image 
of the Cartan domain of the first type Hi := Hi (2, 2) = {x G M. (2x2, C) : ||x|| < 1}, where 
|| • || denotes the operator norm of matrices, under the mapping 7r(x) := (in, £22, det x). 
Note also that E is the image under ix of Hu := Hu (2, 2) := {x G Hi (2, 2) : x = x f } (the 
Cartan domain of the second type). 

In the paper [3J several equivalent definitions of the domain E are given. Recall two of 
them 

(1) E = {(z 1 ,z 2 ,z 3 ) G C 3 : \z 2 - ziz 3 \ + \z x z 2 - z 3 \ + |^i| 2 < 1} 
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and 

(2) E = {(zi, z 2 , z 3 ) E C 3 : \ Zl - z 2 z 3 \ + \z 2 - z x z 3 \ + |z 3 | 2 < 1}. 

It is proven in [3] that the equality between the Caratheodory distance and the Lempert 
function of E with one of the arguments fixed at the origin, 

c*(0, •) = $b(0, •) 

holds on E, which suggests that the equality between both functions holds on E x E. The 
question whether these function are equal was also posed explicitly in |3j. Since both 
functions are biholomorphically invariant, we also get immediately the equality 

c E (z, •) = k E (z, •) on E 

for any z E {^(0) : * E Aut(E)} = {(a,b, ab) : a,b E D}, where Aut(E) is the set of all 
biholomorphisms of E (for a description of holomorphic automorphisms of E see [20] and 

The main purpose of the paper is to show that in fact the above equality holds everywhere 
in the tetrablock thus solving the problem posed in [3]. In other words we prove 

Theorem 1. The equality Ce = k-g, holds. 

Since the tetrablock is a hyperconvex domain (and thus taut), in order to prove the above 
theorem it is equivalent to show that for any /cE-extremal / there is a function F E 0(E, D) 
such that F o f = idp where D denotes the unit disc in C. And this is what we actually 
do. 

Recall that the fundamental Lempert theorem (see [13], [15]) states that the Lempert 
function and the Caratheodory distance coincide on convex domains (and thus on domains 
that may be exhausted by domains biholomorphic to convex ones). Nevertheless, very 
little was known about the converse theorem in a reasonable class of domains (e.g. in 
the class of bounded and pseudoconvex domains). A few years ago C. Costara, J. Agler 
and N. J. Young showed that the Lempert theorem would hold in the symmetrized bidisc 
(see [6], [7], [1]) which is neither biholomorphic to a convex domain (see [7]) nor can be 
exhausted by domains biholomorphic to convex ones (see [S]). The symmetrized bidisc is 
a domain in C 2 denoted by G2 and given by 

(3) G 2 := {(s,p) EC 2 : \s - sp\ + \p 2 \ < 1}. 
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For the basic properties of G2 we refer the reader to 0] . 

Following the ideas in the papers [7J and [8] we show that the tetrablock has the same 
properties. 

Theorem 2. E cannot be exhausted by domains biholomorphic to convex ones. 

The above theorems show that the tetrablock is the second example of that kind. Recall 
also that the symmetrized bidisc is a C-convex domain (see [17]) Therefore it is natural to 
pose the question whether E is biholomorphic to a C-convex domain (or even more, whether 
it can be exhausted by domains biholomorphic to C-convex domains). And we also may 
repeat other open questions concerning the C-convexity. Does the Lempert theorem hold 
for any bounded C-convex domain (see [21])? In fact, since the Lempert Theorem holds 
for all C 2 -smooth C-convex domains (see [H]), the positive answer to this question would 
follow from the positive solution of another problem from [21] : Can any bounded C-convex 
domain be exhausted by C 2 -smooth C-convex domains? 

The tetrablock is an example of a bounded (1, 0, l)-balanced and (0, 1, l)-balanced pseu- 
doconvex domain; recall that if m 1 ,...,m n are non-negative integers (to avoid triviali- 
ties we assume that at least one of m/s is non-zero) then a domain D C C n is called 
(mi, . . . , m n )-balanced if for any z G D and A G B the point (A mi 2:i, . . . , X mn z n ) lies in D. 
In the proof of Theorem [1] we make use of the following result that has a more general 
formulation and is interesting for its own. 

Theorem 3. Let D be an (mi, . . . , m n ) balanced pseudoconvex domain. Assume that ip 
is a complex geodesic in D and ^(A) = {^ mi( Pi{^), ■ ■ ■ , ^ mn( Pn{^)) , A G O, for some tpj 
holomorphic on D, j — 1, . . . , n. 

Then tp G 0{B>, 3D) or if is a complex geodesic in D. 

Note that one may formulate Theorem [3] replacing the geodesies with fc-extremals - in 
this case the proof is immediate. 

Acknowledgments The authors express their thanks to the referee for her/his valuable 
suggestions which essentially improved the quality of the paper. 

2. Definitions, preliminary considerations and proof of Theorem [3] 

First we recall basic definitions of the considered notions. The basic properties in the 
theory of holomorphically invariant functions may be found in [12] . 
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For a domain D C C", w, z G D, we define the Lempert function 

(4) fc D (w, *) := M{p(X h A 2 ) : there is / G 0(B, D), f(X 1 ) = w, /(A 2 ) = 4, 

where p denotes the Poincare distance on D. 

If w 7^ z, then any / as in the definition of k D such that k D (w, z) = p(A 1; A 2 ) is called a 
hp- extremal for (w,z) (or shortly extremal). 

We also define t/je Caratheodory (pseudo) distance by 

(5) c D (w, z) := sup{p(F(w), F(z)) : F G Op, ©)}. 

It is obvious that cp < The Lempert Theorem states that if D is convex then 
kD = c D . 

The idea of the proof of Theorem [T] is the following. We take any fc^-extremal / (and 
the existence of extremal for any pair of different points from E follows from the tautness 
of E) and we show that there is a left inverse to /, i.e. a mapping F G 0(E, B) such that 
F o / is an automorphism (without loss of generality we may require the automorphism to 
be the identity). In such a case the mapping / is called a complex geodesic. There will be 
two kinds of possible choices of F depending on the location of the image of /. Either the 
image of / intersects T := {z G C 3 : Z\Z 2 = z 3 } or it is disjoint from T. In the second case 
we can lift the extremal (using the fact that Trj^w- 1(7-) is a holomorphic covering onto 
E \ T). In the first case we cannot use the lifting coming from the holomorphic covering. 
Nevertheless, making use of the explicit form of the covering we may lift the extremal 
to the extremal lying in TLj. In both cases all the extremals (which are automatically 
complex geodesies because of the convexity of TZj and Tin) are known. So we have a form 
of possible extremals. Now the left inverse will be of two possible forms. Either the one 
considered in several papers in the case the extremal passes through T (see [3] and [TO]) or 
a function obtained from that in a way described in a more general situation in the proof 
of Theorem [3J Therefore, we start with the proof of that theorem. 

Proof of Theorem^ It is clear that ip G G(D,dD) or <p G 0(3, D). Assume that the 
second case holds. Let F G 0(D,3) be such that F o ip = id». 

We claim that for any z = (z\, . . . ,z n ) G D there is exactly one A = X(z) G D such 
that F(X mi zi, . . . , X mn z n ) = X. In fact, fix z G D and consider two functions defined on a 
neighborhood of D: A ->■ F(A mi ^i, . . . , X mn z n ) and A -> A. Since |F(A mi z 1; . . . , X m "z n )\ < 
1 = |A| for all A G 83, the Rouche theorem implies that the function 3 3 X — > X — 
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F(X mi Zi, . . . , \ mn z n ) has exactly one root in D which finishes the proof of our claim. This 
allows us to define well a function G : D i— > D with £r(z) := X(z), z £ D. 
Since the graph of G which is equal to 

(6) {(z, X)eDx3: F{X mi z u X mn z n ) = X} 

is an analytic set (for the notion of analytic sets we refer the reader to [16J) we get that 
G is holomorphic (see e.g. [16], Chapter V, § 1). Moreover, it follows from the definition 
that G o ip(X) = A, A G D, which finishes the proof. □ 

3. Proof of Theorem 1 - the case /(D) nT^0. 

Let Q a (x) = (1 — aa*)~2(x — a)(l — a*:r) _1 (l — a*a)a, a, x G TZi. It is known (see e.g. 
[5]) that $ a G Aut(7£/), $ a (0) = -a and $ a (a) = 0. If additionally X £1X6 symmetric, 
then $ a (x) is symmetric as well. Therefore, $ a G Aut(72.//), a G Tin- 
It follows from [13] that for any ip G Aut(E) there is a $ G Aut(7?.jj) such that 

(7) l/j O 7r(x) = 7T O $(x), XETZjj. 

It is easy to observe that 

(8) $ = 

, , . . . . .' e ie \ / e ie 

for some a = , 01,03 £ D, and U — I or [/ 




;«7 

0, 7/ G R. Direct computations show that the equality ([7j) remains valid on TZj, i.e. 

(9) O 7r(x) = 7T O $(x), X G 7£j. 

Note also that it follows from ([7]) that all automorphisms of E extend holomorphically 
onto a neighborhood of E. 

Put c = I I , where c G D. Let us denote 



c 



'1 - c 



2 xii ^12+cdet x 



(10) ^(x) := $ a (x) = ( V ^l/-^ 1 ^/^gf 1 aM I , x = (x i3 ) G 7^. 



Note that 



v ■ c r 1 - 

1—CX21 v 1 1 1—CX2 



, , , s det x + CX12 
det VM^) - 



1 - cx 2 i 

We start with the following observation: 
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Proof. Using (JTJ we get 

(11) |/2-A/ 3 | + |/i/ 2 -/ 3 | = i-|/i| 2 . 

Let A be such that /(A ) G T. Then |/ 2 (A ) - TTCAoT/a ( A ) | = 1 - |/i(A )| 2 - Using the 
equality /i(Ao)/2(Ao) = /s(Ao) again we infer that 

|/ 2 (A )|(l-|/ 1 (A ) 2 |) = l-|/ 1 (Ao) 2 |, 

whence |/i| = 1 or |/ 2 | = 1 (recall that |/j| < 1, i = 1,2,3). Assume without loss of 
generality that |/i| = 1. Making use of f JTTj) we find that fif^ — fz- D 

Definition 5. For a holomorphic mapping / : D — > E put 

K/)(A)=ord A (/ 1 / 2 -/ 3 ), AgD. 

Remark 6. Note that /(D) C E\ T if and only if z/(/) = 0. Moreover, v is invariant under 
automorphisms of the tetrablock, i.e. 

(12) u(f) = u(<pof), if G Aut(E). 

Actually, it follows from (JTj) that there is an automorphism $ of Tin such that if(n(x)) = 
7r($(x)) for x G Tin- Moreover, $ is of the form (JHJ). Direct calculations show that 
<fi(x)(f 2 (x)—(f 3 (x) = (xiX 2 -x 3 )e 2i( '' ?+e) (l-|a 1 | 2 )(l-|a2| 2 )(l-a 1 a;i-a2X2 + aia2a;3) _2 , x = 
(xx, X2, X3) G E, where n, 9 and a are as in ([8]). Since 1 — a\X\ — 02X2 + 0102X3 = det(l — a*y), 
where y G Tin is such that n(y) = x, we see that the function x h->- 1 — a\X\ —a 2 X2 + 0102X3 
does not vanish on E. This immediately gives (|T2|) . 

Lemma 7. Let / : D — > E be a holomorphic disc such that / _1 (T) 7^ 0. T/ien there is a 
holomorphic disc F : D — >■ 7?./ stzc/i t/iai f = rr o F. 
Moreover, one of two following possibilities holds: 

(a) F(D) C 7^, 

(b) there is an automorphism if of the tetrablock and a holomorphic mapping ip : D — > D 
such that /(A) = ^((0, 0, ^(A))), A G D. 

Proof. Step 1. First consider the case when #/ _1 (T) = 1. Since the group Aut(E) acts 
transitively on 7~, losing no generality we may assume that /(0) = 0. Then there are 
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n, m G N, n + m > 0, such that 

f = (X n 9l ,X m g 2 ,X n+m g 3 ), 

for some holomorphic g = (g u g 2 , g 3 ) : B -> E, gr(0) 7^ 0. Note that gr(D \ {0}) flT = and 
v(g)(0) < i/(/)(0). 

If z/(p)(0) = (i.e. (7(0) G" T), then gi<? 2 — #3 does not vanish on D. Let g be an analytic 

9i 9 



square root of g±g 2 — g 3 - Then the mapping G = ( ) : B — > Tin satisfies g = n o G. 

^ 9 92 

Put F(X) := [ A X ^ I , A G D. Clearly F : B -> 7^7 and / = noF. 

\ X rn g(X) X m g 2 (X) )' 

If v(g)(0) 0, then g(0) G T and, by Lemma H g(D) C E. Let <p G Aut(E) be such 
that <p(g(0)) = 0. There is an analytic disc h : B — > E such that h(0) 7^ and 

^^(A^A^A^/^), 

Tii, m i G N, ?2i + mi > 0. In view of Remark [6] 

v(h)(0) < viip o g)(0) = v(g)(0) < u(f)(0). 

If v (h)(0) = repeating the previous argument we find that there is a mapping H : B — > 
Tin such that h = it o H. Therefore, we may construct a mapping G\ : D — > TZj such 
that tp o g = 7T o Gi. Making use of ([9]) we infer that g = n o G for some analytic disc 
G = (cjij) in 71 j. In particular, / = 7ro.F1, where F\ : B — > TZj is given by the formula 

Fi(X) = ( 9u( ) 912 \ \e ID). If v(h)(0) > we repeat the above procedure 
V A mi ^i(A) X mi g 22 (X) J 

(until u=0). 

Step 2. In the case when f~ l (T) is finite it is sufficient to apply the procedure from the 
previous step to every point of / _1 (T). 

Step 3. Now consider the case when that / _1 (T) is infinite. If /(D) C T, the statement 
is clear. In the other case applying Step 2 to the family of analytic discs /|(i-i/ n )B, n G N, 
we find that there are holomorphic mappings g n : (1 — l/ra)B — > TZj such that 

/ = 7r o g n on (1 — l/n)B. 

Taking a limit of a subsequence we get a holomorphic g : B — > TZi such that 

f = ^°9, 

which finishes the proof of the first assertion. 
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To prove the second statement assume without loss of generality that /(0) = 0. Note 

that o(0) =( I or o(0) =( I for some c 6 ID. If c G <9D we deduce that 

\c J V 00 / 
/ = (0, 0, ip) for some holomorphic mapping ip. In the case when c lies in the unit disc it 

is clear that g(0) G TZj, whence g(H) C TZj. □ 
Recall that any complex geodesic in TZj passing through the origin is of the form 

(13) bb x^u [ X ° I V G Tli 



Z{\) 



where U, V are unitary matrices and Z : D — > D is a holomorphic mapping such that 
either Z(X) = A or \Z(X)\ < |A| for A G IB) \ {0} (see [I]). 

If / is an extremal function in the tetrablock and g : D — > IZi is any holomorphic 
mapping covering / (i.e. 7T o g — /), then ^ is extremal as well. This simple observation 
together with Lemma [7] and the description of complex geodesies of the classical Cartan 
domain of the first type lead to the following statement which is of key importance for our 
considerations: 

Corollary 8. If f : D — > E is an extremal mapping such that /(0) = 0, then either 
/(D) C T or /(A) = (0, 0, e ld X) or there are unitary matrices U, V and there jscGD such 
that 

(M) /(A) = ^(* z J A) )v)), 

where tp c is an automorphism of the Cartan domain of the first type given by the formula 
UDi) and Z : D — » D is a holomorphic mapping. Moreover \Z(X)\ < \X\, X G D \ {0}, or 
Z(X) = X, X G D. 

Lemma 9. Let v = (v^) G dTZj. If Tt{v) G <9E, then |u 12 | = |t>2i|- 

Proof. Seeking a contradiction suppose that |t>i 2 | 7^ 1^21 1- Put u = ( 11 ) , where w 

\ w v 22 J 

is an arbitrary square root of v 12^21- Note that it would suffice to show that 
(15) ||u|| < \\v\\. 

Actually, since ir(v) = ir(v) and \ \v\ \ = 1, the inequality (|T5|) would imply that tt(u) G E. 
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Let us denote p := \\v\\ 2 = p{vv*) and p := \\v\\ 2 = p(vv*). Put d := dett> = detv, 
t ■= tr(vv*) = \v u \ 2 + \v u \ 2 + \v 2 i\ 2 + \v 2 2\ 2 and t := tr(vv*) = \v u \ 2 + 2|t) 12 ||f 2 i| + \v 22 \ 2 . 
It is clear that t < t. 



bmce p = l/2(t + Vt 2 -4d) andp = l/2(f+ v 7 ? -4d), we find that p < p, which proves 
151). □ 



Proof of Theorem^ in the case /(D) fl T 7^ 0. Let / be an extremal mapping in the tetra- 
block such that the image of / intersects the set of triangular points. We lose no generality 
assuming that /(0) = 0. Let r, a e D, r ^ a be such that / is extremal for (/(r), /(c)). 
We aim at showing that / is a complex geodesic. 

If /(A) = (0, 0, e ie A), AG©, the statement is clear. The case /(D) C T follows from [2], 
Corollary 6.9. Therefore, using Corollary [HI we may assume that / is of the form (j!4p . 

First we consider the case when Z(\) = A, A G D. Then /(A) = 7i(ip c (W\)), A G D, 
where W = UV is unitary. Making use of the formula ffTUl) we find that / = (at, ua, 7), 
where or (A) = y/1 - |c| 2 ^nA/(l - c\w 2 i), 7(A) = (det WX 2 + c\w 12 )/(l - cXw 21 ), A G D, 
and uj G <9D is such that w 22 = uwu. Since the tetrablock is (0, 1, rebalanced we may 
assume that ui — 1. 

The descriptions (j2J) of the tetrablock and ([3]) of the symmetrized bidisc give us the 
embedding 

G 2 3 (s,p) h> (s/2,s/2,p) G E. 

Since / is extremal, one can see that / := (2a, 7) is extremal in G 2 . Therefore, it follows 
from [4\ that / is a geodesic in G 2 and its left inverse is given by 



for some a G <9B. Put 



2ap — s . 

F a{s,P) = - , s,p eGj, 

2 — as 



1 — 



where z G D, and recall that |^^| < 1 on E whenever z G D (see [3], Theorem 2.1). It 
follows from the above considerations that \& a (/(A)) = F a (f(X)) = A, A G D, so ^ is a 
left inverse of /, whence / is a complex geodesic. 

Now we focus on the case when |Z(A)| < |A| for A G D \ {0} and c 7^ 0. It is seen 
that there is an open neighborhood D of D and a holomorphic, non-rational mapping 
W : D -> C such that W(B) C D, W(r) = Z(t) and W(a) = Z(a) (note that we do not 
demand W(0) = 0). 
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Put g(A) = 7i(ip r (U | I V)), A G D. Then q is also an extremal function 

v ; v v y W{\) J 

in the tetrablock (as g(a) = /(c), g{r) = /(t)). Therefore g is almost proper, that is 
g*(X) G <9E (g* denotes a nontangential limit of the mapping g) for almost all A G <9D 
w.r.t the Lebesgue measure on the unit circle (see e.g. |9j). Since g is holomorphic in a 
neighborhood of D, the almost properness means that g(8W) C <9E. 
It follows from Lemma M that 

\Xu 2 xVxx + W(X)u 22 v 2 x — c| = \XunVi2 + W (X)u\2V22 + cXW (X) det C/ det V|, 

A G <9D. We claim that there are finite Blaschke products Bx, B 2 such that |-Bi(0)| + 
(52(0)1 ^ and 

(16) S 1 (A)(Au 21 i;i 1 + W{X)u 2 2V2i - c) = B 2 (\)(\u u v 12 + W{X)u 12 v 2 2 + cXW{X)e ie ), 

AG©, where e ie = det U det V. 

To prove the existence of such Blaschke products take any fx, f 2 G 0(D) not vanishing 
identically and satisfying |/i| = |/ 2 | on 3D. Put H(X) := (A — Ai) • ■ • (A — A at), X E D, where 
Ai, . . . , Ajv are common roots of fx and / 2 lying in D and counted with multiplicities. Since 
I A I — I/2 1 on dH) we see that fx/H and f 2 /H do not vanish on dB>. Therefore, there are 
finite Blaschke products B iy i = 1, 2, with no common zeros such that that Fi := fi/(HBi) 
is holomorphic on a neighborhood of D and does not vanish there, i = 1, 2. Since \Bi\ = 1 
on 83 we get that \Fx \ = \F 2 \ on 83. From this we immediately get that F 2 /Fx is constant 
- apply the maximum principle to Fx/F 2 and F 2 /Fx- Let F 2 = 00 Fx, where \u\ = 1. Then 
fx = FxHBx, F 2 = uFxHB 2 , and Bx, B 2 have no common zeros. Putting Bx '■— ooB 2 and 
B 2 := Bx we obtain Blaschke products satisfying the desired claim. 

Since W is non-rational we infer that 

(17) Bx(X)Xu 2l Vxi - cBx(X) = B 2 {X)XuxxVx2, 

Bx(X)u 22 v 21 = B 2 (X)ux2V22 + B 2 (X)cXe id , A G D. 

Putting A = we get Bx(0) = 0. Since B 2 (0) 7^ we get that UX2V22 = 0. 

If Ux2 = 0, then u 2 x = and \uxx\ = \u 22 \ = 1. Putting it to (IT?]) and taking |A| = 1 
we find that |t>i 2 | = |c|. Easy computations give: \vxx\ = \v 22 \ = ^/l — |c| 2 and |f 2 i| = |c|. 
We want to show that / is a complex geodesic. Note that making use of the fact that 
the tetrablock is (1, 0, 1)- and (0, 1, l)-balanced it suffices to get the statement under the 
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additional assumption that un = U22 = 1- Using similar argument we see that it is enough 
to consider that case Vn = a/1 — |c| 2 and Vyi — \c\- Then 



V 



1 - c 



2 



|2 



—u\c\ wyl — \c\ 

for some a; from the unit circle. Replacing Z with uZ we may clearly assume that u = 1. 
Now, after some simple calculations one can deduce that 



l + c|c|Z(A)' l + c|c|Z(A) ' l + c|c|Z(A). 
Therefore / is a complex geodesic (it just of the form (2) in Theorem 2 in pU], with 
permuted two first variables, U2 = 1, u>i G <9D such that c = — u>i\c\, C = |c| 2 and 
= (oJiZ(X) — |c| 2 )/(l — \c\ 2 uiZ{\))). If V22 = we proceed similarly. 
Let us focus on the case c = and |Z(A)| < |A|, A G D \ {0}. First note that we may 
assume that Z extends holomorphically to a neighborhood of D. Let h be a holomorphic 
function in a heighborhood of D such that Z(X) = Xh(X), h(H)) C D. Replacing Z with a 
non-rational W : D — )■ D (holomorphic on a neighborhood of D) such that M^(cr) = Z(a) 
and VT(r) = Z(r), making use of Lemma M and repeating the argument with Blaschke 
products f|T7|) we find that l^i^nl = l^ii^l an d 1^22^21 1 = ^12^22 1- Since U and V are 



unitary we deduce from these equalities that 
(18) \uij\ = \v i:j \, 1 < i,j < 2. 

Put 

jj f X \_ / Amu^h + Z(A)Mi 2 f 2 l AW11U12 + Z(X)U 12 V22 

\ Z(A) y \ A?i 2 lfll + ^(A)«22f21 \U21V 12 + Z(A)'U 2 2f22 

for A lying in some neighborhood of D. Obviously $(D) C 7?./. Define 

AWllfll + Z(A)'Ui 2 f21 A 2 Unf 12 + AZ(A)Ui 2 t>22 
U2lVn + h(X)u 2 2V21 Xu 2 lV 12 + Z(A)u 2 2t>22 

for A from some neighborhood of D. Note that ||^(A)|| = 1 for A G <9B (as \1>\1>* and $<!>* 
have the same eigenvalues on <9B) and ^(0) G TZi. A standard argument implies that \1/ 
maps the unit disc into IZi (apply the maximum principle to the subharmonic function 
log H^(-)ll). Observe that / = tc o \]>, whence \I/ is a complex geodesic in 7£/, as well. 
Denote d = —^21^11 — ^(0)^22^21 • If c' = then u 2 iVn = (remember that \h(0)\ < 1 and 
use the equality |« 2 i^n| = 1^22^21 1), whence U and V are diagonal or anti-diagonal. Then, 



(19) tf(A) 
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it is easy to observe that / is a complex geodesic (more precisely, up to a permutation of 
two fist components the mapping / is of the form /(A) = (uiX, U2Z (X) , U1U2Z (X)) , X G D, 
for some u%,U2 G <9D) . 

If d 7^ 0, then ^21^11 7^ 0. Moving \I/(0) to the origin and making use of the description 
of complex geodesies in TZj we infer that there are unitary matrices Ui, V\ and a mapping 
Z\ defined on D, such that 

(20) *(A) = M^if q Z ° A) W ^» 

and |Zi(A)| < |A| for A G D\ {0} or ^i(A) = A, A G D. Now we are in a position that allows 
us to apply the cases already solved (either Z(X) = A for A G D or c ^ and |Z(A)| < |A| 
for A G D \ {0}). □ 

4. Proof of Theorem 1 - the case /(D) n T = 0. 

Let / : D — > E be an extremal such that /(D) flT= 0. Then there exists a geodesic 
/ : D — > 1Z 1 1 such that / = vr o /. Making use of the form of automorphisms of E 
without loss of generality we may assume that / is a /cE-extremal for (/(0),/(a)) and 
/(0) = (0,0,-/3 2 ). 

Any complex geodesic in 1Z n passing through the origin can be written as 

(21) 

where U is a unitary matrix and Z : D — > D is a holomorphic mapping such that Z(0) = 0. 
Moreover, \Z(X)\ < |A|, A G D \ {0}, or Z(X) = X (see [I]). Assume that 




(22) U 
where \a\ 2 + \b\ 2 = \c\ 2 + |<i| 2 = 1 and ac + bd = 0. After some simple calculations we get 

(23) <f(X) = 



a 2 X + b 2 Z(X) acX + bdZ{X) 
acX + bdZ{X) c 2 X + d 2 Z{X) 



Put A(X) = a 2 X + b 2 Z(X), B(X) = acX + bdZ(X), and C(A) = c 2 + d 2 Z(X). We "move r 
now this geodesic to ( ) and get the following 
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Proposition 10. Let f : D — > E be an extremal mapping for (/(0), f{o~)) such that f(0) = 
(0, 0, -/5 2 ) and /(D) nT = 0. T/ien there exist a,b, c, d e D tuit/j |a| 2 + |6| 2 = |c| 2 + |d| 2 = 1 
and ac + 6<i = sitc/i t/iai 

r9 ^ fm _ ( A{\){l-p) C(A)(l-/3 2 ) A(A)C(A)-(g(A) + /3) 2 ' 

(24) /(A) -^ — W) — ' — W) — ' W) 

where A, B, C are defined as above and A(A) = (1 + (3B(X)) 2 - A(X)C(\)(3 



2 



We show that under the above assumptions the extremal / has its left inverse. First 
note that the following equations are satisfied: \a\ = \d\, \b\ = \c\ and \ad — bc\ = 1. 
Note also that we may always assume that Z(X) = fi\ for some < 1. Actually, if the 
considered mapping is extremal with some Z as above then it will also be extremal with 
Z(X) = fiX where /i = Z'(0). If the new considered extremal intersects T then in view of 
the previous considerations we already know that it is a complex geodesic. Therefore, we 
lose no generality assuming that the extremal omitting T is the one with Z(X) = fiX. 

We want to get some relations on the numbers a, 6, c, d and Z (equivalently, (i) that 
allow us to describe the mappings as in Proposition [TUJ 

When does the equality /1/2 = ^3 hold at some point of D (in other words we want to 
see when /(D) n T =0)? 



(25) 

which is equivalent to 



AC(l-f3 2 ) 2 AC-(B + /3f 



A 2 A 



(26) ACp = {B + P){l + pB). 
Consequently, 

(27) (3XZ(ad - be) 2 - (1 + f3 2 )(Xac + Zbd) -0 = 0. 

Recall that the Cohn criterion (see e.g. [18] ) gives that the equation a^X 2 + aiX + a 2 = 
has both solutions in C \ D iff \a 2 \ > \a \ and |aoai — a 2 (ii\ < \ |a | 2 — | a-2 1 2 1 - 
When we apply it to our situation (Z(X) = /xA) we get that / is as desired iff 

(28) (1 + (3 2 )\fi(ad -bc) 2 (ac + fibd) + (ac + pbd)\ </?(!- |/i| 2 ). 
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Then elementary calculations give that the last inequality (remember about the existing 
relations) is equivalent to |c||d|(l + (3 2 ) < (3. 

It is sufficient to show that we have the left inverse under the sharp inequality 

In view of Theorem [3] it is sufficient to show that for some 7 G D and |r| = 1 the function 

(29) g : D 3 A ^ ( T Az^-/ 1 (A) l / a (A) > r^^/ 8 (A)) G E 

is a geodesic. 

Let F(z) := z z 3 ~ * 2 , 2 G E. We shall prove that by the proper choice of r and 7 the 
function h := F o g is an automorphism of D. But it is sufficient, by the Schwarz-Pick 
Lemma to show that |/i'(0)| = 1 - \h(0)\ 2 . 

But /i(0) = -T7/3 2 and 



(30) /i'(0) = r/3 2 (l - | 7 | 2 ) - 2r/3 7 (ac + bdp){l - (3 2 ) + 

(c 2 + d 2 p)(l - /3 2 ) + (c 2 + rf 2 /i)(l - /3 2 ) + r 2 7 2 (a 2 + b 2 p){l - (5 2 )(3 2 . 

Consequently, 

(31) h'{0) = (1 - /3 2 )((c - r/3 7 a) 2 + - br^) 2 ) + r/3 2 (l - | 7 | 2 ). 

We choose \r\ = 1 and 7 G D such that d = 6r/?7 and \h'(0)\ = |(1 - (3 2 )\c - r^a\ 2 + 
/3 2 (1 — I7I 2 ), which is possible under the assumption |<i| 2 < |&| 2 /3 2 , which is equivalent 
= |c| and |c| 2 + \d\ 2 = 1) to j^jp < |c| 2 . And the last inequality is equivalent to 
\p\V\ - \ r \ 2 ) < P 2 

M \ L M ) ^ (1+^2)2- 

5. E CANNOT BE EXHAUSTED BY DOMAINS BIHOLOMORPHIC TO CONVEX ONES 

In this Section we prove Theorem [21 

For z G C 3 put p(z) := max ||(7r|7j, rj .) _1 (z)||. The properness of ir\n n implies that p is 
plurisubhar monic . 

Proof of Theorem^ For any e G (0, 1) we define G e := {z G C 3 : p(z) < 1 — e}. Assume 
that U e is a neighborhood of G e and f e :U € ^ V e where V e is a convex domain. Without 
loss of generality we may assume that G V e , V t is a convex domain, / e (0) = 0, f' e (0) = id. 
Denote <f\(z) := (Xz%, Xz 2 , A 2 z 3 ), A G C, z G E. 

Fix u> = (iwi, w 2 , W3), z = (zi, Z2, z 3 ) G C 3 and r G [0, 1]. Put 
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(1) R := max{p(w),p(z)}, 

(2) g e {\) := f-\rf e {^{w)) + (1 - r)f t {<p x {z))). 

Note that g e (0) = and that g e is well-defined for |A| < (l—e)/R. Moreover, p(g t (X)) < 1 
for any |A| < (1 - e)/R. Put h e {\) := ip 1/x {g e {\)). Then h e : O(0, (1 - e)/R) \ {0} ^ C 3 is 
a holomorphic mapping. Then simple calculations show the following properties 

(1) (g e )' j (0)=rw j + (l-r)z j , J = 1,2, 

(2) (fc)i(0) = 0. 

Consequently, h e extends holomorphically to 0. More calculations show that 



dz\ 

^^(O)r(l - r)(w 2 - z 2 f + 2 |^(0)r(l - r)( Wl - Zl )(w 2 - z 2 ). 



(32) (fc)S(O) = 2(rw 3 + (1 - r)z 3 ) + -^(0)r(l - r)( Wl - Zl ) 2 + 



Define 



Then 



(34) h e (0) = (rwi + (1 - r)z 1 ,rw 2 + (1 - r)z 2l rw 3 + (1 - r)^ 3 + s e r(l - r)(wi - 2i) 2 + 

t e r(l - r)(u> 2 - ^ 2 ) 2 + u e r(l - r)(wi - z 1 )(w 2 - z 2 )). 

By the maximum principle 

(35) p(h € (X)) = P ( Vl/x (g e (X)) = jrPiffeW) < r^- 



Hence, p(h £ (0)) < 

Our next aim is to show that 



(36) lims e = limt e = limM e = 0. 

<e->0 <=->0 £ ->0 

Note that the equalities fl36|) imply that p{rw + (1 — r)z) < max{p(w), p(z)} for all 
w,z6 C 3 , which contradicts the non-convexity of E. 
We are just left with the proof of the above equalities. 
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Put r = „-. For the proof of the convergence of s e consider two points w = (1,1,1), 
z = (-1, 1, -1). Putting them to dMD and flM} we find that p(0, 1, s e ) < 1/(1 - e). This 
inequality implies that s e — > 0. 

Similarly, putting u> = (1, 1, 1) and z = (1, —1, —1) one can show that t e — > 0. 

Finally, taking z = ((, £, C) and w = (—(, — C, C)j where |£| = 1 is such that = |w e | we 
find that p(0, 0, £(1 + (t t + (s e + \u e \)) < 1/(1 — e). Making use of just proven two equalities 
we get the equality lim^oM,, = 0. □ 
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